Order statistic

 ( Nonparametric Statistics ) 

:$X\_\{(i)\}\; \backslash stackrel\{d\}\{=\}\; \backslash frac\{1\}\{\backslash lambda\}\backslash left(\; \backslash sum\_\{j=1\}^i\; \backslash frac\{Z\_j\}\{n-j+1\}\; \backslash right)$

where the Z _j are iid standard exponential random variables (i.e. with rate parameter 1). This result was first published by Alfréd Rényi.

$u=F\_X(x)$

and

$du=f\_X(x)\backslash ,dx$

to derive the following probability density functions for the order statistics of a sample of size n drawn from the distribution of X:

$f\_\{X\_\{(k)\}\}(x)\; =\backslash frac\{n!\}\{(k-1)!(n-k)!\}F\_X(x)^\{k-1\}1-F\_X(x)^\{n-k\}\; f\_X(x)$

$f\_\{X\_\{(j)\},X\_\{(k)\}\}(x,y)\; =\; \backslash frac\{n!\}\{(j-1)!(k-j-1)!(n-k)!\}F\_X(x)^\{j-1\}F\_X(y)-F\_X(x)^\{k-1-j\}1-F\_X(y)^\{n-k\}f\_X(x)f\_X(y)$ where $x\backslash le\; y$

$f\_\{X\_\{(1)\},\backslash ldots,X\_\{(n)\}\}(x\_1,\backslash ldots,x\_n)=n!f\_X(x\_1)\backslash cdots\; f\_X(x\_n)$ where $x\_1\backslash le\; x\_2\backslash le\; \backslash dots\; \backslash le\; x\_n.$

As an example, consider a random sample of size 6. In that case, the sample median is usually defined as the midpoint of the interval delimited by the 3rd and 4th order statistics. However, we know from the preceding discussion that the probability that this interval actually contains the population median is

$\{6\backslash choose\; 3\}(1/2)^\{6\}\; =\; \{5\backslash over\; 16\}\; \backslash approx\; 31\backslash \%.$

Although the sample median is probably among the best distribution-independent of the population median, what this example illustrates is that it is not a particularly good one in absolute terms. In this particular case, a better confidence interval for the median is the one delimited by the 2nd and 5th order statistics, which contains the population median with probability

$\backslash left\{6\backslash choose(1/2)^\{6\}\; =\; \{25\backslash over\; 32\}\; \backslash approx\; 78\backslash \%.$

With such a small sample size, if one wants at least 95% confidence, one is reduced to saying that the median is between the minimum and the maximum of the 6 observations with probability 31/32 or approximately 97%. Size 6 is, in fact, the smallest sample size such that the interval determined by the minimum and the maximum is at least a 95% confidence interval for the population median.

$U\_\{(\backslash lceil\; np\; \backslash rceil)\}\; \backslash sim\; AN\backslash left(p,\backslash frac\{p(1-p)\}\{n\}\backslash right).$

For a general distribution F with a continuous non-zero density at F ⁻¹( p), a similar asymptotic normality applies:

$X\_\{(\backslash lceil\; np\; \backslash rceil)\}\; \backslash sim\; AN\backslash left(F^\{-1\}(p),\backslash frac\{p(1-p)\}\{nf(F^\{-1\}(p))^2\}\backslash right)$

where f is the density function, and F ⁻¹ is the quantile function associated with F. One of the first people to mention and prove this result was Frederick Mosteller in his seminal paper in 1946. Further research led in the 1960s to the Bahadur representation which provides information about the errorbounds. The convergence to normal distribution also holds in a stronger sense, such as convergence in relative entropy or KL divergence.M. Cardone, A. Dytso and C. Rush, "Entropic Central Limit Theorem for Order Statistics," in IEEE Transactions on Information Theory, vol. 69, no. 4, pp. 2193-2205, April 2023, doi: 10.1109/TIT.2022.3219344.

An interesting observation can be made in the case where the distribution is symmetric, and the population median equals the population mean. In this case, the sample mean, by the central limit theorem, is also asymptotically normally distributed, but with variance σ² /n instead. This asymptotic analysis suggests that the mean outperforms the median in cases of low kurtosis, and vice versa. For example, the median achieves better confidence intervals for the Laplace distribution, while the mean performs better for X that are normally distributed.

$B(k,n+1-k)\backslash \; \backslash stackrel\{\backslash mathrm\{d\}\}\{=\}\backslash \; \backslash frac\{X\}\{X\; +\; Y\},$

where

$X\; =\; \backslash sum\_\{i=1\}^\{k\}\; Z\_i,\; \backslash quad\; Y\; =\; \backslash sum\_\{i=k+1\}^\{n+1\}\; Z\_i,$

with Z_i being independent identically distributed exponential random variables with rate 1. Since X/ n and Y/ n are asymptotically normally distributed by the CLT, our results follow by application of the delta method.

The mutual information between uniform order statistics is given by

I(U_{(r)}; U_{(m)}) = T_{m-1} + T_{n-r} - T_{m-r+1} - T_n
     

The expected value of the first order statistic $Y\_\{(1)\}$ given a sample of $N$ total observations yields,

$E(Y\_\{(1)\})\; =\; \backslash frac\{1\}\{(N+1)\; g(0)\}\; +\; \backslash frac\{1\}\{(N+1)(N+2)\}\; \backslash int\_\{0\}^\{1\}\; Q\text{'}\text{'}(z)\; \backslash delta\_\{N+1\}(z)\; \backslash ,\; dz$

where $Q$ is the quantile function associated with the distribution $g\_\{Y\}$, and $\backslash delta\_N(z)\; =\; (N+1)(1-z)^N$. This equation in combination with a jackknifing technique becomes the basis for the following density estimation algorithm,

  Input: A sample of $N$ observations. $\backslash \{x\_\backslash ell\backslash \}\_\{\backslash ell=1\}^M$ points of density evaluation. Tuning parameter $a\; \backslash in\; (0,1)$ (usually 1/3).
  Output: $\backslash \{\backslash hat\{f\}\_\backslash ell\backslash \}\_\{\backslash ell=1\}^M$ estimated density at the points of evaluation.
     

  1: Set $m\_N\; =\; \backslash operatorname\{round\}(N^\{1-a\})$
  2: Set $s\_N\; =\; \backslash frac\{N\}\{m\_N\}$
  3: Create an $s\_N\; \backslash times\; m\_N$ matrix $M\_\{ij\}$ which holds $m\_N$ subsets with $s\_N$ observations each.
  4: Create a vector $\backslash hat\{f\}$ to hold the density evaluations.
  5: '''for''' $\backslash ell\; =\; 1\; \backslash to\; M$ '''do'''
  6:     '''for''' $k\; =\; 1\; \backslash to\; m\_N$ '''do'''
  7:         Find the nearest distance $d\_\{\backslash ell\; k\}$ to the current point $x\_\backslash ell$ within the $k$th subset
  8:      '''end for'''
  9:      Compute the subset average of distances to $x\_\backslash ell:d\_\backslash ell\; =\; \backslash sum\_\{k=1\}^\{m\_N\}\; \backslash frac\{d\_\{\backslash ell\; k\}\}\{m\_N\}$
 10:      Compute the density estimate at $x\_\backslash ell:\backslash hat\{f\}\_\backslash ell\; =\; \backslash frac\{1\}\{2\; (1+\; s\_N)\; d\_\backslash ell\}$
 11:  '''end for'''
 12: '''return''' $\backslash hat\{f\}$
     

In contrast to the bandwidth/length based tuning parameters for histogram and kernel based approaches, the tuning parameter for the order statistic based density estimator is the size of sample subsets. Such an estimator is more robust than histogram and kernel based approaches, for example densities like the Cauchy distribution (which lack finite moments) can be inferred without the need for specialized modifications such as IQR based bandwidths. This is because the first moment of the order statistic always exists if the expected value of the underlying distribution does, but the converse is not necessarily true.

The cumulative distribution function of the $k^\backslash text\{th\}$ order statistic can be computed by noting that

Similarly,

Note that the probability mass function of $X\_\{(k)\}$ is just the difference of these values, that is to say

For example, a recent application in actuarial risk can be found in, where some weighted premium principles in terms of record claims and kth record claims are provided.

• Rankit
• Box plot
• BRS-inequality
• Concomitant (statistics)
• Fisher–Tippett distribution
• Bapat–Beg theorem for the order statistics of independent but not necessarily identically distributed random variables
• Bernstein polynomial
• L-estimator – linear combinations of order statistics
• Rank-size distribution
• Selection algorithm

• Sample maximum and minimum
• Quantile
• Percentile
• Decile
• Quartile
• Median
• Mean
• Sample mean and covariance

• Retrieved Feb 02, 2005
• Retrieved Feb 02, 2005
• C++ source Dynamic Order Statistics